Real-time multi-view 3D reconstruction for interventional environments

This thesis addresses the topic of real-time multi-view 3D reconstruction in interventional environments. Due to the special working conditions in interventional rooms this is a very challenging topic. However, the ability to recover the 3D structure of a dynamic scene in real-time is essential for many novel and innovative interventional applications. In this thesis we take a step towards this goal by presenting the design of a multi-view reconstruction system taking into consideration the conditions typically encountered in interventional environments. Two installations of the system were created: One in a laboratory environment and one in a real interventional room. While the laboratory system was used for development and evaluation purposes, the hospital system was used to learn about the conditions which have to be taken into account in a clinical setting. These conditions which typically include a cluttered environment with complex lighting and occlusions are discussed in detail and recommendations are given for dealing with them. Based on the proposed system several applications were developed to demonstrate the benefits of real-time 3D reconstruction in interventional environments. In a first application the real-time reconstruction is used to ensure the safe operation of automated medical devices such as C-arms by detecting potential collisions between the device and its environment (staff, patient, utilities) in advance. A second application is aimed at estimating the physician’s radiation exposure by tracking his reconstruction and accumulating the radiation received by each body part over time. The promising experimental results obtained using these applications show the possible impact of our work for the medical community and illustrate the clinical value of multi-camera systems in the development of intelligent, integrated interventional suites. This is underlined by the fact that the proposed system has already been used by other researches for recovering the workflow of surgical procedures [PMW+09]. During the course of this thesis the topic of reconstruction and organization of large image collections was also investigated. This research area is gaining importance since it has become easy and affordable to acquire large image collections due to the proliferation of digital cameras. However, the huge amount of images available poses new challenges for existing algorithms. The focus of this part of the thesis therefore lies on the efficient use of large image collections. Three related problems are investigated, namely obtaining high quality reconstructions using graph-cut methods, increasing the reconstruction efficiency by clustering image collections and building a unified framework for various applications related to image organization, such as image-based navigation, image set reduction and scene summarization.